Papers with deep understanding of the
Continual Learning of Large Language Models (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | This tutorial explores the challenges of continual learning in large language models . participants will learn strategies to mitigate forgetting and manage data and evaluation pipelines . |
| Approach: | This tutorial offers a comprehensive exploration of continual learning in the context of large language models. |
| Outcome: | This tutorial explores the challenges of continual learning in large language models . participants will learn how to manage data and evaluation pipelines and adapt responsibly . |
NavHint: Vision and Language Navigation Agent with a Hint Generator (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing work on vision and language navigation relies on navigation-related losses to establish the connection between vision and modalities, neglecting aspects of helping the navigation agent build a deep understanding of the visual environment. |
| Approach: | They propose to provide indirect supervision to the navigation agent through a hint generator that generates visual descriptions during navigation. |
| Outcome: | The proposed method improves the navigation performance and interpretability of the R2R and R4R datasets. |
FlowDelta: Modeling Flow Information Gain in Reasoning for Conversational Machine Comprehension (D19-58)
Copied to clipboard
| Challenge: | Existing machine comprehension models focus on a single-turn setting and do not account for previous reasoning processes. |
| Approach: | They propose to explicitly model the information gain through the dialogue reasoning . they propose to apply the proposed mechanism to other machine comprehension models . |
| Outcome: | The proposed model achieves state-of-the-art performance in a conversational QA dataset QuAC and a sequential instruction understanding dataset SCONE. |
Improving Biomedical Abstractive Summarisation with Knowledge Aggregation from Citation Papers (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models struggle to generate technical summaries that are on par with those produced by biomedical experts due to the lack of domain-specific background knowledge. |
| Approach: | They propose a attention-based citation aggregation model that integrates domain-specific knowledge from citation papers and a large-scale biomedical summarisation dataset to build on. |
| Outcome: | The proposed model outperforms state-of-the-art approaches and achieves substantial improvements in biomedical abstractive summarisation. |
MReD: A Meta-Review Dataset for Structure-Controllable Text Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | a new text generation dataset is needed to controllable text summarization, but it lacks the domain knowledge. |
| Approach: | They propose to use existing text generation datasets to leverage input and control signals . they propose to annotate each meta-review sentence manually with a control signal . |
| Outcome: | The proposed method can be used to control the structure of a text generation dataset . it can be applied to a variety of tasks, including a task with a large number of meta-review sentences . |
ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning (2021.acl-long)
Copied to clipboard
Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, Jie Zhou
| Challenge: | Existing pre-training objectives do not explicitly model relational facts in text . Experimental results show that ERICA can improve typical PLMs on several language understanding tasks, including relation extraction, entity typing and question answering. |
| Approach: | They propose a contrastive learning framework ERICA to obtain a deep understanding of entities and relations in text. |
| Outcome: | The proposed framework can improve PLMs on several language understanding tasks, especially under low-resource settings. |
Autoencoding Keyword Correlation Graph for Document Clustering (2020.acl-main)
Copied to clipboard
| Challenge: | Existing representation learning models do not capture the intra-sentential and inter-sententential features of long-text. |
| Approach: | They propose a graph-based representation for document clustering that builds a Graph Autoencoder on a Keyword Correlation Graph. |
| Outcome: | The proposed graph autoencoder can achieve better clustering performance than existing features. |
Graph-Assisted Culturally Adaptable Idiomatic Translation for Indic languages (2025.findings-acl)
Copied to clipboard
| Challenge: | a single source idiom can have multiple target-language equivalents depending on cultural references and contextual variations. |
| Approach: | They propose an adaptive graph neural network-based method that learns intricate mappings between idiomatic expressions and generalizes to both seen and unseen nodes during training. |
| Outcome: | The proposed method improves translation quality even in resource-constrained settings, facilitating improved idiomatic translation in smaller models. |
Combinatory Grammar Tells Underlying Relevance among Entities (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches focus on dependencies among words while paying limited attention to other types of syntactic structure. |
| Approach: | They propose an alternative approach that takes advantage of combinatory categorial grammar to detect the relation between entities. |
| Outcome: | The proposed model performs state-of-the-art on two widely used English benchmark datasets. |
What is More Likely to Happen Next? Video-and-Language Future Event Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models cannot make multimodal commonsense predictions of future events based on video and dialogue . |
| Approach: | They propose a task to predict which event is more likely to happen in a video clip . they use a dataset with 28,726 future event prediction examples from 10,234 videos . |
| Outcome: | The proposed model provides a good starting point but leaves room for future work. |
Towards Robustifying NLI Models Against Lexical Dataset Biases (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show that deep learning models exploit dataset biases without deep understanding of the language semantics. |
| Approach: | They propose two methods to debiase models against lexical dataset biases . they use contradiction-word bias and word-overlapping bias as examples . |
| Outcome: | The proposed method removes label bias at embedding level, while the other uses a bag-of-words sub-model to capture features likely to exploit the bias. |
BLM-s/lE: A structured dataset of English spray-load verb alternations for testing generalization in LLMs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Current NLP models are achieving performance comparable to human capabilities on well-established benchmarks. |
| Approach: | They propose a BLM task to identify a missing element in a linguistic pattern from a list of candidate options based on a given matrix. |
| Outcome: | The proposed framework is based on the spray-load verb alternations in English as a case study. |
End-to-end Aspect-based Sentiment Analysis with Combinatory Categorial Grammar (2023.findings-acl)
Copied to clipboard
| Challenge: | End-to-end aspect-based sentiment analysis (EASA) is a natural language processing task that requires a deep understanding of the running text. |
| Approach: | They propose a method to improve EASA with CCG supertags that carry syntactic and semantic information of the associated words. |
| Outcome: | The proposed approach outperforms baselines and achieves state-of-the-art results on all datasets. |
PARROT: Zero-Shot Narrative Reading Comprehension via Parallel Reading (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to narrative comprehension require extensive annotation of data. |
| Approach: | They propose a zero-shot approach for narrative comprehension through parallel reading using two parallel narratives that tell the same story. |
| Outcome: | The proposed approach surpasses previous zero-shot approaches and comparable performance to fully supervised models. |
UniDebugger: Hierarchical Multi-Agent Framework for Unified Software Debugging (2025.emnlp-main)
Copied to clipboard
Cheryl Lee, Chunqiu Steven Xia, Longji Yang, Jen-tse Huang, Zhouruixing Zhu, Lingming Zhang, Michael R. Lyu
| Challenge: | Existing LLMs focus on isolated steps and struggle with complex bugs. |
| Approach: | They propose a framework for unified debugging through multi-agent synergy . it mimics the entire cognitive processes of developers with each agent specialized as a particular component of this process . |
| Outcome: | The proposed framework outperforms state-of-the-art methods on repo-level benchmarks. |
Infogen: Generating Complex Statistical Infographics from Documents (2025.acl-long)
Copied to clipboard
| Challenge: | Existing efforts to generate simple charts have focused on generating simple infographics from text-heavy documents. |
| Approach: | They propose to generate statistical infographics composed of multiple sub-charts that are contextually accurate, insightful, and visually aligned. |
| Outcome: | The proposed framework outperforms both open-source and closed LLMs in text-to-statistical infographic generation. |
GRASP: Graph-Reasoning Aided Survey Planning for High-Fidelity Related Work Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Graph-Reasoning Aided Survey Planning (GRASP) is a framework for analyzing cited papers. |
| Approach: | They propose a framework that combines LLM planning for related work generation with graph algorithms to extract key relationships among cited papers. |
| Outcome: | The proposed framework generates RWS that closely match human-written targets in terms of discourse roles, intents, and grouping of citations. |